Skip to content

feat(ui): unify five confidence vocabularies onto the canonical band ladder - #56

Merged
slittycode merged 5 commits into
mainfrom
fix/unified-confidence
May 17, 2026
Merged

feat(ui): unify five confidence vocabularies onto the canonical band ladder#56
slittycode merged 5 commits into
mainfrom
fix/unified-confidence

Conversation

@slittycode

Copy link
Copy Markdown
Owner

Stacked on #55#54 → main. PR base is `fix/citation-headline`. When #55 merges, GitHub auto-retargets to `main` and the diff stays clean.

Summary

Audit finding #4Confidence is shown in five different vocabularies. This PR promotes the existing four-band ladder (`apps/ui/src/services/sessionMusician/confidenceBand.ts`) to the canonical primitive across every confidence surface in the results page. Producers now see one mental model of trust (Solid scaffold / Workable draft / Rough sketch / Unreliable) instead of five competing systems.

Frontend-only. Gemini still emits `HIGH/MED/LOW` enum strings; the UI converts locally via a new `toConfidenceBand` normalizer.

Why

`PURPOSE.md`'s chain-of-custody invariant requires low-confidence measurements to produce visibly hedged advice. Today the producer sees the same trust level expressed four different ways depending on which card they're looking at — "MED" on a characteristics card, "Moderate" on a notes chip, "CONF 62%" on the Key card, "SCORE 0.86" on the Tempo card. Five vocabularies for one signal makes the hedge legible nowhere. After this PR, every confidence reads in the same language: a colored pill with band label + percent.

What changed

New primitives

  1. `toConfidenceBand(value)` in `confidenceBand.ts` — single normalizer that accepts:

    1. Numeric 0-1 floats (Phase 1 canonical)
    2. Numeric 0-100 integers
    3. Gemini string enums (`HIGH/MED/LOW` + `High/Moderate/Low` variants)
    4. Percent strings (`"62%"`)
      Returns the matching `ConfidenceBand` or `null` for unparseable input. String enums map to band-mid values (HIGH→0.9, MED→0.6, LOW→0.3) so `formatBandPillLabel` reads as honest hedges.
  2. `ConfidenceBandBadge` variant + band-override props — `variant: 'full' | 'compact'` (compact omits the copy paragraph for card-corner placement; full is the Session Musician panel default, unchanged) and `band?: ConfidenceBand` (skip the round-trip when the caller pre-converted).

Five vocabularies retired

Vocabulary Sites New rendering
V1: `HIGH/MED/LOW` pills Detected Characteristics cards `<ConfidenceBandBadge variant="compact" band={toConfidenceBand(item.confidence)} />`
V2: `High/Moderate/Low` chips Confidence Notes viewModel returns `{ label, band }`; chip renders `<ConfidenceBandBadge variant="compact" band={badge.band} />`
V3: `CONF X%` text Key card, Character card, alternate Key card in MeasurementDashboard `<ConfidenceBandBadge variant="compact" confidence={...} />`
V4: `SCORE X.XX` badge Tempo card in AnalysisResults, Tempo card in MeasurementDashboard `<ConfidenceBandBadge variant="compact" confidence={...} />`

Cross-Check ✓/✗ in MeasurementDashboard preserved unchanged — that's an agreement signal (do multiple BPM detectors agree?), orthogonal to confidence. The audit's framing grouped it loosely with the confidence vocabularies but it's structurally different.

Dead code retired

`normalizeConfidenceLevel`, `parseConfidenceScalar`, `confidenceClass`, two copies of `formatBpmScore`. The `ConfidenceLevel` type and `MelodyInsightsViewModel.confidenceLabel` are kept alive — they drive non-pill key/value text rows in the Sonic Element melody insights (flagged for follow-up migration).

Intentional refinement

The old three-level `normalizeConfidenceLevel` mapped scalar 0.5-0.79 AND "medium" both to "Moderate". The new normalizer maps "medium" to 0.6 → workable (consistent) and scalar 0.25-0.49 to "Rough sketch" — a more granular hedge than the old "Low" bucket. Documented in the test rewrite.

Test plan

  • `npx vitest run tests/services/confidenceBand.test.ts` — 9 new cases for `toConfidenceBand` (null/undefined fallback, 0-1 floats, 0-100 integers, all string enum variants, percent strings, NaN/Infinity, negative clamping).
  • `npx vitest run tests/services/confidenceBandBadge.test.ts` — 4 new cases for the compact variant and the `band` override (label-only when no confidence, tone override, Gemini-string routing).
  • `npx vitest run tests/services/analysisResultsViewModel.test.ts` — rewrote the existing `toConfidenceBadges` shape assertion; added a `band: null` fallback case.
  • `npx vitest run tests/services/analysisResultsUi.test.ts` — 5 new integration cases proving each V1-V4 site renders band pill text and not the old vocabularies.
  • `npm run verify` — full gate green: lint + ~600 unit tests + production build + 44 Playwright smoke (+2 skipped: live Gemini, live backend).
  • Live UI check — drove the upload against saved run `855f1376-ec1e-4e3e-b145-bbf6eb479296` via Playwright POST intercept. Probe results:
    1. 36 band pills visible across the page: 21 Solid scaffold, 7 Workable draft, 5 Rough sketch, 3 Unreliable.
    2. Zero `CONF X%` leakage — all three V3 sites converted.
    3. The two remaining `SCORE X` matches in the page text are unrelated ("TEXTURE SCORE", "HEALTH SCORE" — MixDoctor signals, not confidence).
    4. V1 visual — Detected Characteristics cards show the chain-of-custody hedge working as designed: 4 cards render SOLID SCAFFOLD (green), Detuned Supersaw renders WORKABLE DRAFT (orange) — the same vocabulary as every other confidence surface.
    5. V3 visual — top metric cards (Tempo, Key, Character) show `SOLID SCAFFOLD · 93%` / `SOLID SCAFFOLD · 100%` pills replacing the old `CONF X%` text and `SCORE` badges.
    6. Cross-Check ✓/✗ is structurally preserved; the saved run doesn't populate `bpmAgreement` so it doesn't render, but the code path is untouched.

Reviewer notes

  1. `MelodyInsightsViewModel.confidenceLabel` kept. This field drives a key/value text row inside Sonic Element melody cards ("High (78%)" style) — not a confidence pill, so out of scope for this PR. Flagged as a follow-up to migrate to the band vocabulary.
  2. `characteristicPillClass` kept alive. It's still used by an unrelated chip on the Character metric card (characteristic-name chips with color-encoded confidence). That's a different presentation than the V1 Detected Characteristics pill; out of scope. Flagged as a follow-up to unify.
  3. 3-level → 4-band mismatch is intentional refinement. The old normalizer collapsed Rough into Low. The new normalizer surfaces it as its own band so producers get a more granular hedge.
  4. Cross-Check preserved. It's an agreement signal, not confidence. Visually adjacent to the new band pill on the Tempo card.
  5. Smoke specs grep-clean — no smoke test asserts on the retired vocabulary strings.

🤖 Generated with Claude Code

slittycode and others added 5 commits May 16, 2026 19:08
Closes four orientation gaps in CLAUDE.md: a top-of-Architecture map
that flags advisory/, experiments/, docs/, and tests/ground_truth/ as
off-path; a Scripts-at-a-glance section covering scripts/ and
apps/backend/scripts/; a pointer from the Phase 3 paragraph to
SamplePlayback.tsx + sampleGenerationClient.ts; and a curl example
plus allowlisted field paths on the csv_export.py module entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Oz <oz-agent@warp.dev>
Removes four classes of broken/placeholder content that made the Phase 2
recommendations read as engine output instead of producer-facing advice
(audit finding #1).

1. Mix Chain card role text — `buildRoleSentence` used to fabricate
   "{stage phrase} by {verb}" by lowercasing the first letter of
   Gemini's `reason`. Because `reason` is a present-tense clause, every
   card produced ungrammatical splices like "Controls bass energy by
   ensures the extreme low-end mono…". Now renders the reason verbatim
   with a capitalized first letter and trailing period; HIGH-END cue
   suffix preserved as "(for …)".

2. Patch Framework `patchRole` — was a 7-key category-keyed fallback
   (`mapPatchRole`) that stamped duplicated placeholders ("Primary tone
   generator" on every SYNTHESIS card). Removed the field, the
   fallback, the JSX paragraph, and the contribution to the
   `inferProcessingGroup` text-concat. The category chip + per-card
   `whyThisWorks` already carry the bucket and explanation.

3. Patches/Mix Chain overlap — `buildPatchCards` now case-insensitively
   filters out devices that also appear in `mixAndMasterChain` so the
   Patches section stops re-listing chain devices. Synthetic fallbacks
   (Stereo Width, MIDI Clip Guide) are built from Phase 1 only and
   bypass the filter intentionally.

4. Interpretation Caution raw-JSON dump — the panel rendered
   `originalValue` verbatim, which for dropped recommendations is a
   JSON-dumped `AbletonRecommendation` (see `_stringify_warning_value`
   in `server_phase2.py`). Added `formatDroppedValue` helper that
   parses JSON-shaped values and renders a compact "device: X ·
   parameter: Y · value: Z" summary. Non-JSON strings pass through;
   invalid JSON falls back to a truncated raw string. Also resolves
   the `$Saturator` symptom — Gemini's hallucinated `$`-prefixed
   device name now surfaces as a readable summary line instead of
   leaking through a raw JSON dump.

5. System Diagnostics dev-only leak — `validateNewFieldCoverage`
   emits "Phase 1 field 'X' is present… this warning is benign"
   coverage signals meant for the engine team, not producers. Added
   optional `audience?: 'dev' | 'user'` to `ValidationViolation`,
   marked `NEW_FIELD_UNCITED` as dev, and updated
   `Phase2ConsistencyReport` to filter dev-audience violations from
   the rendered table AND from the header counts (so the header
   doesn't read "5 warnings shown" above an empty table). The
   underlying `ValidationReport` still carries every violation for
   tests and offline analysis.

Tripwire note: the backend grammar fix
`_apply_phase2_grammar_fixes` in `server_phase2.py` only runs on the
legacy `/api/phase2` endpoint, not the analysis-runs path the UI
uses. After this change it becomes a redundant no-op against the new
render shape; left untouched.

Tests: 16 new unit cases across viewModel, UI rendering, validator,
and consistency report. `npm run verify` passes (lint + 565 unit
tests + build + 44 smoke tests).

Live UI verified against saved run 855f1376… — all nine text-leak
probes (`by ensures`, `by ducks`, `by generates`,
`Primary tone generator`, `Texture and movement stage`,
`Stereo placement stage`, raw `\$Saturator`, `this warning is
benign`, "is present in the measurement payload but no Phase 2…")
return false.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Audit Finding #3 — "the chain-of-custody citation, the product's whole
differentiator, renders as a footnote."

Adds a one-line `CitationHeadline` primitive inside the collapsed header
of every Mix Chain / Patch / Sonic Element card so producers see the
measurement evidence at scan-time without expanding. The expanded
CitationBlock stays in the body unchanged — this is the collapsed-state
companion, not a replacement.

Each headline reads `{label} {value} →` with the arrow pointing into
the device h4/h3 that follows in the title row, making the implicit
"measurement justified device" statement visible — exactly the audit's
literal example "Crest factor 8.2 dB → Glue Compressor."

Confidence sibling pills (Solid / Workable / Rough / Unreliable, same
ladder as ConfidenceBandBadge) ride alongside the value when the
primary field has a paired *Confidence sibling. This preserves the
chain-of-custody invariant: low-confidence measurements visibly hedge
in the collapsed view too, not just after expansion.

Implementation:
1. `CitationHeadline` added as a sibling export in
   `apps/ui/src/components/CitationBlock.tsx`. Shares the file's
   imports, the `CONFIDENCE_PILL_CLASSES` map, and the audit lineage.
   Composes existing pure helpers (`pickPhase1Value`,
   `formatCitedValue`, `humanizeFieldPath`,
   `pickPhase1Confidence`, `getConfidenceBand`,
   `formatBandPillLabel`) — no new resolver logic.
2. Mount points in `AnalysisResults.tsx`:
   - Mix Chain card header (~line 2056): between the title row and
     the role paragraph.
   - Patch card header (~line 2215): between the title row and the
     MetaBadgeList.
   - Sonic Element card header (~line 1912): between the title row
     and the summary paragraph.
3. Each mount is guarded by `card.phase1Fields.length > 0`. Cards
   without cited fields fall back to today's exact layout.

Live verification against saved run 855f1376… (the audit reproducer):
all 10 Mix Chain cards, 2 Patch cards, and 7 Sonic Element cards
render the headline correctly with appropriate confidence pills
(PUMPING STRENGTH 47% [ROUGH SKETCH 30%], SUPERSAW DETECTED
[WORKABLE DRAFT 78%], ACID BASS DETECTED [SOLID SCAFFOLD 99%],
NOTE TRANSCRIPTION 61% [WORKABLE DRAFT 61%], KEY C Minor
[SOLID SCAFFOLD 93%], TEMPO 145 BPM [SOLID SCAFFOLD 100%]).

Mobile (375px) verified — headlines fit, pills stay aligned, no
overflow.

Tests: 15 new cases — 10 in `tests/services/citationHeadline.test.ts`
(mirrors `citationBlock.test.ts`) covering label/value/arrow render,
null fallback, confidence-pill bands, value formatter parity,
typography hooks. 5 in `analysisResultsUi.test.ts` covering Mix Chain
/ Patch / Sonic integration, empty-fields fallback, and the
low-confidence Unreliable-band path.

`npm run verify` passes: lint + 575 unit tests + build + 44 smoke.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ladder

Audit Finding #4 — the producer used to see five competing vocabularies
for "how much should I trust this number?" — HIGH/MED/LOW pills on
Detected Characteristics cards, High/Moderate/Low chips on Confidence
Notes, bare CONF X% text on Key/Character metric cards, SCORE X.XX
badges on the BPM card (two render paths), and the canonical four-band
ladder used only by the Session Musician panel. The chain-of-custody
invariant in PURPOSE.md requires low-confidence measurements to produce
visibly hedged advice — five vocabularies fragment that signal so the
producer can't build a single mental model of trust.

This PR promotes the existing four-band ladder
(apps/ui/src/services/sessionMusician/confidenceBand.ts) to the canonical
primitive across every confidence surface. Frontend-only; Gemini still
emits HIGH/MED/LOW and the UI converts locally via a new
`toConfidenceBand` normalizer.

What changed:

1. New normalizer `toConfidenceBand(value)` in `confidenceBand.ts`
   accepts numeric 0-1 floats, 0-100 integers, Gemini string enums
   (HIGH/MED/LOW + High/Moderate/Low variants), and percent strings
   ("62%"). Returns the matching `ConfidenceBand` or null for
   unparseable input. The string enums map to band-mid values (HIGH→0.9,
   MED→0.6, LOW→0.3) so `formatBandPillLabel` reads as honest hedges.

2. `ConfidenceBandBadge` gains:
   - `variant: 'full' | 'compact'` — `compact` omits the copy paragraph
     so the pill can sit inline in card corners and metric-card footers.
     `full` (default) preserves the Session Musician panel behavior
     unchanged.
   - `band?: ConfidenceBand` override prop — when supplied, skips the
     `getConfidenceBand` round-trip. Useful when the caller pre-converts
     a string enum.

3. V1: Detected Characteristics cards (AnalysisResults.tsx ~1650) —
   bespoke HIGH/MED/LOW ternary pill replaced with `<ConfidenceBandBadge
   variant="compact" band={toConfidenceBand(item.confidence)} />`.
   `characteristicPillClass` helper kept alive only for the unrelated
   characteristic-name chips on the Character metric card (line 1000);
   flagged as a follow-up.

4. V2: Confidence Notes chips (AnalysisResults.tsx ~1187) —
   `toConfidenceBadges` viewModel return shape changed from
   `{ label, level }` (3-level legacy enum) to `{ label, band }`
   (canonical four-band ladder). `confidenceClass` helper deleted.

5. V3: Plain `CONF X%` text (3 sites) — replaced with
   `<ConfidenceBandBadge variant="compact" confidence={...} />` in:
   - AnalysisResults.tsx Key card footer
   - AnalysisResults.tsx Character card footer
   - MeasurementDashboard.tsx alternate Key card

6. V4: `SCORE X.XX` badges (2 sites) — replaced with band pills in
   AnalysisResults.tsx Tempo card and MeasurementDashboard.tsx Tempo
   card. `formatBpmScore` deleted from both files. Cross-Check ✓/✗
   StatusBadge in MeasurementDashboard preserved — that's an agreement
   signal (do multiple BPM detectors agree?), not confidence.

7. Dead code retired: `normalizeConfidenceLevel`,
   `parseConfidenceScalar`, `confidenceClass`, two copies of
   `formatBpmScore`. `ConfidenceLevel` type and
   `MelodyInsightsViewModel.confidenceLabel` kept alive — they drive
   non-pill key/value text rows in Sonic Element melody insights
   (flagged for follow-up migration).

3-level → 4-band mismatch documented as an intentional refinement: the
old `normalizeConfidenceLevel` mapped scalar 0.5-0.79 AND "medium" both
to "Moderate"; the new normalizer maps "medium" to 0.6 → workable
(consistent), and scalar 0.25-0.49 to "Rough sketch" (more granular
than the old "Low" bucket).

Tests: 20 new cases across 4 files —
1. `confidenceBand.test.ts` — 9 cases for `toConfidenceBand`
   (null/undefined fallback, 0-1 floats, 0-100 integers, all string
   enum variants, percent strings, NaN/Infinity, negative clamping).
2. `confidenceBandBadge.test.ts` — 4 cases for the compact variant
   and the `band` override prop (label-only when no confidence, tone
   override, Gemini-string routing).
3. `analysisResultsViewModel.test.ts` — rewrote the existing
   `toConfidenceBadges` shape assertion; added a `band: null`
   fallback case.
4. `analysisResultsUi.test.ts` — 5 integration cases proving each
   V1-V4 site renders band pill text and not the old vocabularies.

Live UI verified against saved run 855f1376… — 36 band pills visible
across the page (21 Solid scaffold, 7 Workable draft, 5 Rough sketch,
3 Unreliable). Zero `CONF X%` leakage. Detected Characteristics cards
show the chain-of-custody hedge working as designed (Detuned Supersaw
renders WORKABLE DRAFT in orange while the other 4 detections render
SOLID SCAFFOLD in green). Cross-Check ✓/✗ structurally preserved (the
saved run doesn't populate bpmAgreement, so it doesn't render, but the
code path is untouched).

`npm run verify` passes: lint + ~600 unit + build + 44 smoke.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE

Summary

Frontend-only UI consolidation: five competing confidence vocabularies (HIGH/MED/LOW pills, High/Moderate/Low chips, CONF X%, SCORE X.XX, numeric scalars) unified onto the existing four-band ladder via a new toConfidenceBand normalizer and a compact variant of ConfidenceBandBadge. Phase 1 schema untouched, no backend changes. All 4 modified test files pass cleanly (128 tests); the 3 suite-level failures are pre-existing decision_gate.*.live.test.ts tests that require a Gemini API key and are unrelated to this PR.

Findings

Worth considering — semantic divergence between toConfidenceBand and getConfidenceBand for values in (1, 100]

toConfidenceBand(1.88) returns unreliable (treats 1.88 as a percentage, divides → 0.0188). ConfidenceBandBadge confidence={1.88} calls getConfidenceBand(1.88) directly and returns solid (1.88 ≥ 0.8 threshold). The PR's own test fixture uses bpmConfidence = 1.88, so this out-of-range value is a real Phase 1 possibility.

The current callers are architecturally correct — Phase 1 numeric values go through getConfidenceBand directly, Phase 2 strings go through toConfidenceBand — so this doesn't manifest as a bug today. But the toConfidenceBand comment says it accepts "numeric 0-1 floats (Phase 1 canonical)" without warning that Phase 1 values occasionally exceed 1.0. A future caller who passes phase1.bpmConfidence through toConfidenceBand (reasonable mistake given the docstring) silently gets the wrong band. Worth one sentence: "Phase 1 values that occasionally exceed 1.0 — e.g. bpmConfidence — should be passed directly to getConfidenceBand or as the confidence prop on ConfidenceBandBadge; routing them through toConfidenceBand misclassifies them as sub-1% values."

Worth considering — band + confidence conflict renders misleading pill text

When both props are supplied and they disagree — band=unreliable, confidence=0.95 — the pill reads "Unreliable · 95%". The test covers this and labels it "exotic combination." Not a blocker given every actual call site uses one or the other, never both conflicting. A discriminated union on the props would make the footgun impossible, but that's a follow-up scope item.

Test results

616 pass / 3 fail / 1 skipped. The 3 failures are pre-existing live-API tests (decision_gate.multi.live, decision_gate.real.live, decision_gate.stems.live) that require GEMINI_API_KEY. No regression from this PR.

Phase boundary check

Clean. Phase 1 values (bpmConfidence, keyConfidence, genreDetail.confidence) are read at render sites and passed as the confidence prop — no mutation, no re-derivation. Phase 2 string enums go through toConfidenceBand for display normalization only. Phase1Result and the HTTP envelope are untouched.


Generated by Claude Code

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE (self-review — GitHub prevents approve on own PRs)

Summary

Promotes the existing four-band confidence ladder to the canonical primitive across every confidence surface (five vocabularies → one). New toConfidenceBand() normalizer handles 0-1 floats, 0-100 integers, Gemini string enums (HIGH/MED/LOW), and percent strings. ConfidenceBandBadge gains variant: 'compact' | 'full' and an optional band prop. ConfidenceBadgeViewModel.level: ConfidenceLevel replaced with band: ConfidenceBand | null. Frontend-only.

Findings

Worth considering:

  • toConfidenceBand uses substring matching: lower.includes('low') maps "low" to rough, but also matches strings like "below", "slow", "fellow". In practice the inputs are controlled (Gemini enums + numeric strings) so this is harmless, but exact-match for the enum strings would be safer.
  • ConfidenceBandBadge with both confidence and band undefined falls through to getConfidenceBand(0) → unreliable. The dev-mode console.warn catches it in development. Production behavior (silent unreliable pill) is reasonable. The // eslint-disable-next-line no-console comment is cargo-culted — no ESLint is configured in this project, so the suppression is a no-op.
  • MelodyInsightsViewModel.confidenceLabel (three-level ConfidenceLevel) is explicitly deferred. Fine for now; the follow-up is tracked.
  • No CI check runs on head commit 2bba3372. Local test suite: 616 pass on fix/quick-hits.

Test results

No CI on head commit. 9 new confidenceBand.test.ts cases + 4 new confidenceBandBadge.test.ts cases + updated analysisResultsViewModel.test.ts + 5 integration UI cases. Coverage is thorough: null/undefined, boundary values, all string enum variants, percent strings, NaN/Infinity, negative clamping, compact variant.

Phase boundary check

Clean. All changes are display-layer. ConfidenceBadgeViewModel is a view model type, not a Phase 1 contract type. Phase 1 confidence scalars (bpmConfidence, keyConfidence, genreDetail.confidence) are read, never assigned.


Generated by Claude Code

slittycode added a commit that referenced this pull request May 17, 2026
…esh (#57), nightly audit doc (#59)

* audits: nightly 2026-05-16 — 5 issues (4 test failures + 1 review item)

3 vitest decision_gate.*.live.test.ts failures (missing /tmp snapshot
gating), 1 Playwright upload-estimate-phase1.spec.ts timeout, and one
mixDoctor.estimatePlr fallback flagged for human boundary review. No
Phase 1 ground-truth mutations found.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(claude): add repo-layout, scripts map, Phase 3 UI entry, CSV recipe

Closes four orientation gaps in CLAUDE.md: a top-of-Architecture map
that flags advisory/, experiments/, docs/, and tests/ground_truth/ as
off-path; a Scripts-at-a-glance section covering scripts/ and
apps/backend/scripts/; a pointer from the Phase 3 paragraph to
SamplePlayback.tsx + sampleGenerationClient.ts; and a curl example
plus allowlisted field paths on the csv_export.py module entry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: add Layer 2 transcription evaluation guide

Co-Authored-By: Oz <oz-agent@warp.dev>

* fix(ui): stop Phase 2 results surface from leaking engine output

Removes four classes of broken/placeholder content that made the Phase 2
recommendations read as engine output instead of producer-facing advice
(audit finding #1).

1. Mix Chain card role text — `buildRoleSentence` used to fabricate
   "{stage phrase} by {verb}" by lowercasing the first letter of
   Gemini's `reason`. Because `reason` is a present-tense clause, every
   card produced ungrammatical splices like "Controls bass energy by
   ensures the extreme low-end mono…". Now renders the reason verbatim
   with a capitalized first letter and trailing period; HIGH-END cue
   suffix preserved as "(for …)".

2. Patch Framework `patchRole` — was a 7-key category-keyed fallback
   (`mapPatchRole`) that stamped duplicated placeholders ("Primary tone
   generator" on every SYNTHESIS card). Removed the field, the
   fallback, the JSX paragraph, and the contribution to the
   `inferProcessingGroup` text-concat. The category chip + per-card
   `whyThisWorks` already carry the bucket and explanation.

3. Patches/Mix Chain overlap — `buildPatchCards` now case-insensitively
   filters out devices that also appear in `mixAndMasterChain` so the
   Patches section stops re-listing chain devices. Synthetic fallbacks
   (Stereo Width, MIDI Clip Guide) are built from Phase 1 only and
   bypass the filter intentionally.

4. Interpretation Caution raw-JSON dump — the panel rendered
   `originalValue` verbatim, which for dropped recommendations is a
   JSON-dumped `AbletonRecommendation` (see `_stringify_warning_value`
   in `server_phase2.py`). Added `formatDroppedValue` helper that
   parses JSON-shaped values and renders a compact "device: X ·
   parameter: Y · value: Z" summary. Non-JSON strings pass through;
   invalid JSON falls back to a truncated raw string. Also resolves
   the `$Saturator` symptom — Gemini's hallucinated `$`-prefixed
   device name now surfaces as a readable summary line instead of
   leaking through a raw JSON dump.

5. System Diagnostics dev-only leak — `validateNewFieldCoverage`
   emits "Phase 1 field 'X' is present… this warning is benign"
   coverage signals meant for the engine team, not producers. Added
   optional `audience?: 'dev' | 'user'` to `ValidationViolation`,
   marked `NEW_FIELD_UNCITED` as dev, and updated
   `Phase2ConsistencyReport` to filter dev-audience violations from
   the rendered table AND from the header counts (so the header
   doesn't read "5 warnings shown" above an empty table). The
   underlying `ValidationReport` still carries every violation for
   tests and offline analysis.

Tripwire note: the backend grammar fix
`_apply_phase2_grammar_fixes` in `server_phase2.py` only runs on the
legacy `/api/phase2` endpoint, not the analysis-runs path the UI
uses. After this change it becomes a redundant no-op against the new
render shape; left untouched.

Tests: 16 new unit cases across viewModel, UI rendering, validator,
and consistency report. `npm run verify` passes (lint + 565 unit
tests + build + 44 smoke tests).

Live UI verified against saved run 855f1376… — all nine text-leak
probes (`by ensures`, `by ducks`, `by generates`,
`Primary tone generator`, `Texture and movement stage`,
`Stereo placement stage`, raw `\$Saturator`, `this warning is
benign`, "is present in the measurement payload but no Phase 2…")
return false.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(ui): surface primary citation in collapsed card headers

Audit Finding #3 — "the chain-of-custody citation, the product's whole
differentiator, renders as a footnote."

Adds a one-line `CitationHeadline` primitive inside the collapsed header
of every Mix Chain / Patch / Sonic Element card so producers see the
measurement evidence at scan-time without expanding. The expanded
CitationBlock stays in the body unchanged — this is the collapsed-state
companion, not a replacement.

Each headline reads `{label} {value} →` with the arrow pointing into
the device h4/h3 that follows in the title row, making the implicit
"measurement justified device" statement visible — exactly the audit's
literal example "Crest factor 8.2 dB → Glue Compressor."

Confidence sibling pills (Solid / Workable / Rough / Unreliable, same
ladder as ConfidenceBandBadge) ride alongside the value when the
primary field has a paired *Confidence sibling. This preserves the
chain-of-custody invariant: low-confidence measurements visibly hedge
in the collapsed view too, not just after expansion.

Implementation:
1. `CitationHeadline` added as a sibling export in
   `apps/ui/src/components/CitationBlock.tsx`. Shares the file's
   imports, the `CONFIDENCE_PILL_CLASSES` map, and the audit lineage.
   Composes existing pure helpers (`pickPhase1Value`,
   `formatCitedValue`, `humanizeFieldPath`,
   `pickPhase1Confidence`, `getConfidenceBand`,
   `formatBandPillLabel`) — no new resolver logic.
2. Mount points in `AnalysisResults.tsx`:
   - Mix Chain card header (~line 2056): between the title row and
     the role paragraph.
   - Patch card header (~line 2215): between the title row and the
     MetaBadgeList.
   - Sonic Element card header (~line 1912): between the title row
     and the summary paragraph.
3. Each mount is guarded by `card.phase1Fields.length > 0`. Cards
   without cited fields fall back to today's exact layout.

Live verification against saved run 855f1376… (the audit reproducer):
all 10 Mix Chain cards, 2 Patch cards, and 7 Sonic Element cards
render the headline correctly with appropriate confidence pills
(PUMPING STRENGTH 47% [ROUGH SKETCH 30%], SUPERSAW DETECTED
[WORKABLE DRAFT 78%], ACID BASS DETECTED [SOLID SCAFFOLD 99%],
NOTE TRANSCRIPTION 61% [WORKABLE DRAFT 61%], KEY C Minor
[SOLID SCAFFOLD 93%], TEMPO 145 BPM [SOLID SCAFFOLD 100%]).

Mobile (375px) verified — headlines fit, pills stay aligned, no
overflow.

Tests: 15 new cases — 10 in `tests/services/citationHeadline.test.ts`
(mirrors `citationBlock.test.ts`) covering label/value/arrow render,
null fallback, confidence-pill bands, value formatter parity,
typography hooks. 5 in `analysisResultsUi.test.ts` covering Mix Chain
/ Patch / Sonic integration, empty-fields fallback, and the
low-confidence Unreliable-band path.

`npm run verify` passes: lint + 575 unit tests + build + 44 smoke.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(ui): unify five confidence vocabularies onto the canonical band ladder

Audit Finding #4 — the producer used to see five competing vocabularies
for "how much should I trust this number?" — HIGH/MED/LOW pills on
Detected Characteristics cards, High/Moderate/Low chips on Confidence
Notes, bare CONF X% text on Key/Character metric cards, SCORE X.XX
badges on the BPM card (two render paths), and the canonical four-band
ladder used only by the Session Musician panel. The chain-of-custody
invariant in PURPOSE.md requires low-confidence measurements to produce
visibly hedged advice — five vocabularies fragment that signal so the
producer can't build a single mental model of trust.

This PR promotes the existing four-band ladder
(apps/ui/src/services/sessionMusician/confidenceBand.ts) to the canonical
primitive across every confidence surface. Frontend-only; Gemini still
emits HIGH/MED/LOW and the UI converts locally via a new
`toConfidenceBand` normalizer.

What changed:

1. New normalizer `toConfidenceBand(value)` in `confidenceBand.ts`
   accepts numeric 0-1 floats, 0-100 integers, Gemini string enums
   (HIGH/MED/LOW + High/Moderate/Low variants), and percent strings
   ("62%"). Returns the matching `ConfidenceBand` or null for
   unparseable input. The string enums map to band-mid values (HIGH→0.9,
   MED→0.6, LOW→0.3) so `formatBandPillLabel` reads as honest hedges.

2. `ConfidenceBandBadge` gains:
   - `variant: 'full' | 'compact'` — `compact` omits the copy paragraph
     so the pill can sit inline in card corners and metric-card footers.
     `full` (default) preserves the Session Musician panel behavior
     unchanged.
   - `band?: ConfidenceBand` override prop — when supplied, skips the
     `getConfidenceBand` round-trip. Useful when the caller pre-converts
     a string enum.

3. V1: Detected Characteristics cards (AnalysisResults.tsx ~1650) —
   bespoke HIGH/MED/LOW ternary pill replaced with `<ConfidenceBandBadge
   variant="compact" band={toConfidenceBand(item.confidence)} />`.
   `characteristicPillClass` helper kept alive only for the unrelated
   characteristic-name chips on the Character metric card (line 1000);
   flagged as a follow-up.

4. V2: Confidence Notes chips (AnalysisResults.tsx ~1187) —
   `toConfidenceBadges` viewModel return shape changed from
   `{ label, level }` (3-level legacy enum) to `{ label, band }`
   (canonical four-band ladder). `confidenceClass` helper deleted.

5. V3: Plain `CONF X%` text (3 sites) — replaced with
   `<ConfidenceBandBadge variant="compact" confidence={...} />` in:
   - AnalysisResults.tsx Key card footer
   - AnalysisResults.tsx Character card footer
   - MeasurementDashboard.tsx alternate Key card

6. V4: `SCORE X.XX` badges (2 sites) — replaced with band pills in
   AnalysisResults.tsx Tempo card and MeasurementDashboard.tsx Tempo
   card. `formatBpmScore` deleted from both files. Cross-Check ✓/✗
   StatusBadge in MeasurementDashboard preserved — that's an agreement
   signal (do multiple BPM detectors agree?), not confidence.

7. Dead code retired: `normalizeConfidenceLevel`,
   `parseConfidenceScalar`, `confidenceClass`, two copies of
   `formatBpmScore`. `ConfidenceLevel` type and
   `MelodyInsightsViewModel.confidenceLabel` kept alive — they drive
   non-pill key/value text rows in Sonic Element melody insights
   (flagged for follow-up migration).

3-level → 4-band mismatch documented as an intentional refinement: the
old `normalizeConfidenceLevel` mapped scalar 0.5-0.79 AND "medium" both
to "Moderate"; the new normalizer maps "medium" to 0.6 → workable
(consistent), and scalar 0.25-0.49 to "Rough sketch" (more granular
than the old "Low" bucket).

Tests: 20 new cases across 4 files —
1. `confidenceBand.test.ts` — 9 cases for `toConfidenceBand`
   (null/undefined fallback, 0-1 floats, 0-100 integers, all string
   enum variants, percent strings, NaN/Infinity, negative clamping).
2. `confidenceBandBadge.test.ts` — 4 cases for the compact variant
   and the `band` override prop (label-only when no confidence, tone
   override, Gemini-string routing).
3. `analysisResultsViewModel.test.ts` — rewrote the existing
   `toConfidenceBadges` shape assertion; added a `band: null`
   fallback case.
4. `analysisResultsUi.test.ts` — 5 integration cases proving each
   V1-V4 site renders band pill text and not the old vocabularies.

Live UI verified against saved run 855f1376… — 36 band pills visible
across the page (21 Solid scaffold, 7 Workable draft, 5 Rough sketch,
3 Unreliable). Zero `CONF X%` leakage. Detected Characteristics cards
show the chain-of-custody hedge working as designed (Detuned Supersaw
renders WORKABLE DRAFT in orange while the other 4 detections render
SOLID SCAFFOLD in green). Cross-Check ✓/✗ structurally preserved (the
saved run doesn't populate bpmAgreement, so it doesn't render, but the
code path is untouched).

`npm run verify` passes: lint + ~600 unit + build + 44 smoke.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs: refresh stale items against current architecture

CHANGELOG.md "Unreleased" was missing nine shipped features (Phase 3
audition samples, URL ingest, admin DELETE bypass, source-audio route,
CSV export, publicStatus, reassigned spectrogram, chain-of-custody audit
overhaul, BatchedBandpass centralization, hosted runtime foundation,
backend monolith split). Added them.

CLAUDE.md's analyze.py CLI example only listed four flags; expanded to
match the full surface (--standard, --pitch-note-only, --stem-dir,
--stem-output-dir, --pitch-note-backend). Expanded the Frontend key
service files list from 9 to 16 entries to cover the services that
landed during the audit overhaul (httpClient, sampleGenerationClient,
appliedRecommendations + userLabels, phase1Picker + phaseLabels,
audioFile, fieldAnalytics + diagnosticLogs, sessionMusician helpers).

apps/ui/AGENTS.md File Map likewise expanded from 4 to 14 service
entries against the same list.

docs/SAMPLE_GENERATION.md's "Snapshot integration" section claimed an
`AnalysisRunSnapshot.stages.sampleGeneration` shape that does not exist
in apps/ui/src/types/backend.ts — the snapshot tracks only measurement,
pitchNoteTranslation, and interpretation. Replaced the stale TS block
with a description that matches the actual on-demand artifact path
(consistent with the doc's own line 47).

apps/backend/AGENTS.md and apps/backend/ARCHITECTURE.md now flag
symbolic_extract.py as orphaned and broken — it imports a removed
BasicPitchBackend symbol from analyze.py, would raise ImportError at
load, and is not referenced from any other module. Slated for removal.

docs/history/README.md had `library-review-torchfx-2026-05-13.md`
listed twice; removed the duplicate.

docs/ARCHITECTURE_STRATEGY.md "Last updated" stamp bumped to May 2026
with a note that no shifts to the three-layer thesis or library
decisions accompanied the recent feature work.

Date stamps in apps/backend/AGENTS.md and apps/ui/AGENTS.md refreshed
to 2026-05-17.

https://claude.ai/code/session_01QEmFAUJSoRgTqPcFTfcnYW

* fix(ui): audit quick-hits bundle (StickyNav, Mix Chain, Mix Doctor, header)

Closes six items from the audit's "Quick hits (low ROI, real)" list.
Each is independent; bundled into one PR because they're all small
surgical fixes.

1. QH1 — StickyNav clipped "MEASUREMENTS" at 1440px. Was
   overflow-x-auto + min-w-max, which produced an invisible horizontal
   scroll (macOS hides scrollbars by default; last pill looked clipped
   with no scroll affordance). Switched to flex-wrap so pills flow onto
   multiple rows at narrow widths; whitespace-nowrap prevents intra-
   label wrap.

2. QH2 — Mix Chain card-number order badges removed. Cards are grouped
   by processing stage AFTER ordering, so the order numbers appeared
   out-of-sequence within each group ("1, 6, 8, 9 / 2, 4 / 5, 7 / 3 /
   10"), which read as a presentation bug. The visual sequence within
   each group is already meaningful; the badge added confusion without
   information. Dropped.

3. QH4 — Mix Doctor Band Diagnostics table now shows the target
   range alongside the optimal. Previously the column rendered only
   the optimal dB (e.g. "-22.0"), so two bands with similar Delta dB
   could land in different Issue buckets without the producer being
   able to see why. The Issue is determined by absolute thresholds
   (target.minDb / maxDb), not by the diff from optimal. Showing the
   range makes the verdict legible at a glance — Sub Bass -6.4 norm,
   range -16 to -8, exceeds the upper bound by 1.6 dB → TOO-LOUD;
   Low Mids -14.5 norm, range -26 to -14, sits at the boundary →
   OPTIMAL. Threshold logic itself was correct; this is a display fix.

4. QH5 — "Dense DAW Lab" link removed from the header. A prior audit
   pass had de-emphasized it to muted text, but the new audit re-
   flagged it for sitting in the primary flow's header without context
   for what it is. Route stays accessible via direct URL (the
   getAppViewHref helper is preserved for future re-introduction
   behind a settings menu, and main.tsx still routes to the
   DenseDawConcept component when activeView === 'daw-concept').

5. QH6 — header model selector visually demoted. Dropped the
   "Interpretation Model" label and shrank the styling from a bordered
   dropdown to a discreet text-secondary inline select. The audit
   flagged the prior styling as foregrounding an AI-model choice an
   intermediate producer has no basis to make. Selector stays
   accessible (smoke spec requires it visible at desktop viewport;
   `responsive-layout.spec.ts` and `upload-estimate-phase1.spec.ts`
   both guard the `phase2-model-desktop` testid) but now reads as
   background metadata. Aria-label + title attribute carry the long-
   form context for assistive tech and hover discoverability.

6. QH7 — Mix Doctor section title renamed from "MixDoctor" to
   "Mix Doctor". The audit flagged that the section header rendered
   as "Mixdoctor" with a lowercase 'd', inconsistent with every other
   section name. Root cause: `formatWord` in `utils/displayText.ts`
   case-normalizes single-token CamelCase to sentence case, so
   "MixDoctor" was rendering as "Mixdoctor". Two words ("Mix Doctor")
   matches every other section name's pattern (Style Profile, Project
   Setup, Track Layout, Mix & Master Chain, Patch Framework, etc.)
   and renders cleanly through the existing pipeline.

`npm run verify` passes: lint + ~600 unit + build + 44 smoke. Live UI
verified against saved run 855f1376… with probes confirming all six
fixes:
- StickyNav.overflow-x-auto wrapper gone (pills wrap to 2 rows)
- Zero Mix Chain order badges in #section-mix-chain
- "Target (range)" header present, "Target dB" header gone
- "Dense DAW Lab" text absent from page
- "Interpretation Model" label absent from page
- "Mix Doctor" section title present, "Mixdoctor" absent

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Oz <oz-agent@warp.dev>
Base automatically changed from fix/citation-headline to main May 17, 2026 21:29
@slittycode
slittycode merged commit 4d9c1f8 into main May 17, 2026
@slittycode
slittycode deleted the fix/unified-confidence branch May 17, 2026 21:29
slittycode added a commit that referenced this pull request May 17, 2026
… item (#60)

Two post-merge follow-ups from the review of bc79247:

1. Confidence-band unification (audit #56 follow-up): drop ConfidenceLevel
   enum and the duplicate hardcoded thresholds in buildMelodyInsights;
   render Sonic Element melody row and Melody Patch parameter row via
   the canonical getConfidenceBand label. Threshold definition lives in
   one place. Added positive test that averageConfidence: 0.83 lands on
   the Solid scaffold band, plus two render-site tests covering the
   actual interpolated strings.

2. estimatePlr boundary review (audit 2026-05-16 §4): keep the fallback,
   document why. Backend's analyze_plr emits the same value byte-for-byte
   in both --fast and full paths; UI fallback only fires on legacy
   snapshots predating the field and lands on the identical number.
   Inline comment + audit-doc resolution subsection so the next nightly
   doesn't re-flag this.

Verified: tsc clean, 605/605 vitest pass, build green, 654/654 backend
tests pass. CI green on Frontend and Backend.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant